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ABSTRACT 

Autonomous  mobile  robots  typically  require  a  preconceived  and  very  detailed  navigational  model  (map) 
of  their  intended  operating  environment,  but  most  law  enforcement  and  urban  warfare  response  scenarios 
preclude  the  availability  of  such  a  priori  information.  ROBART  ni  is  an  advanced  demonstration 
platform  for  non-lethal  robotic  response  measures,  incorporating  a  supervised  autonomous  navigation 
system  specifically  configured  to  support  minimally  attended  operation  in  previously  unexplored  interior 
structures.  A  “human-centered  mapping”  strategy  has  been  developed  to  ensure  valid  first-time 
interpretation  of  navigational  landmarks  as  the  robot  builds  its  world  model.  The  accuracy  of  the  robot’s 
real-time  position  estimation  (and  hence  the  model  itself)  is  significantly  enhanced  by  an  innovative 
algorithm  which  exploits  a  heuristic  that  assumes  the  majority  of  man-made  structures  are  characterized 
by  parallel  and  orthogonal  walls. 

Intruder  detection  and  assessment  capabilities  are  supported  by  intelligent  fusion  of  data  collected  by  a 
multitude  of  various  sensors.  Initial  detection  is  by  a  360-degree  array  of  eight  passive-infrared  motion 
detectors  responding  to  the  thermal  energy  gradient  created  by  a  moving  human  target,  with  partial 
validation  from  a  Doppler  microwave  motion  detector.  The  robot’s  head- mounted  sensors  (and  non- 
lethal  weapon  system)  are  then  panned  to  the  center  of  any  perceived  disturbance  for  further  assessment. 
Automatic  tracking  of  any  confirmed  movement  is  accomplished  through  conventional  image  processing 
using  a  black-and-white  video  surveillance  camera  equipped  with  a  near-infrared  illuminator  for  low- 
light  conditions. 

The  non-lethal  response  systems  incorporated  on  ROBART  HI  include  a  six -barreled  pneumatically- 
powered  Gatling  gun  capable  of  firing  a  variety  of  3/ 16-inch-diameter  projectiles,  including  simulated 
tranquilizer  darts.  A  visible-red  laser  sight  is  provided  to  facilitate  supervised  operation  as  well  as 
manual  control  of  the  weapon  using  remote  video  relayed  to  the  operator  from  the  robot’s  head-mounted 
camera.  Three  ear-piercing  103-decibel  sirens  can  be  activated  to  alert  those  nearby  of  imminent  danger 
while  simultaneously  disorienting  a  confirmed  intruder. 

For  increased  effectiveness,  a  distributed  master/slave  sensor  network  can  be  instantiated  by  deploying  a 
number  of  small  slave  robots  that  follow  ROBART  HI  into  an  unexplored  structure  in  trailing  convoy 
fashion.  Slaves  are  deployed  and  recovered  automatically,  so  the  remote  operator  needs  to  provide 
supervisory  control  of  only  the  master.  The  slave  robots  can  perform  a  multitude  of  assigned  functions, 
to  include  acting  as  a  distributed  video  and  motion  sensor  network,  in  addition  to  providing  a  seamless 
RF-relay  capability  throughout  the  building. 
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1.  BACKGROUND 


From  a  navigational  perspective,  the  type  of  control  strategy  employed  on  a  mobile  platform  runs  the  full 
spectrum  defined  by  teleoperated  at  the  low  end  through  fully  autonomous  at  the  upper  extreme.  A 
teleoperated  machine  of  the  lowest  order  has  no  onboard  intelligence  and  blindly  executes  the  drive  and 
steering  commands  sent  down  in  real  time  by  a  remote  operator.  A  fully  autonomous  mobile  platform, 
on  the  other  hand,  keeps  track  of  its  position  and  orientation  and  typically  uses  some  type  of  world 
modeling  scheme  to  represent  the  location  of  perceived  objects  in  its  surroundings. 

A  very  common  modeling  approach  is  to  employ  a  statistical  certainty-grid  representation  (Moravec, 
1985),  where  each  cell  in  the  grid  corresponds  to  a  particular  unit  square  of  floor  space  (i.e.,  a  three-inch 
square,  a  six-inch  square,  depending  on  the  desired  map  resolution).  The  numerical  value  assigned  to 
each  cell  represents  the  probability  that  its  associated  location  in  the  building  is  occupied  by  some 
object,  with  a  value  of  zero  indicating  free  space  (i.e.,  no  obstacles  present).  Additional  “special-case” 
values  can  be  reserved  in  this  fashion  to  encode  other  specific  features  or  attributes  in  the  model  (i.e., 
doors,  walls),  as  will  be  discussed  later. 

The  existence  of  an  absolute  world  model  allows  for  automatic  path  planning  and  execution,  and  for 
subsequent  route  revisions  in  the  event  a  new  obstacle  is  encountered.  Unfortunately,  the  autonomous 
execution  of  indoor  paths  generally  requires  some  a  priori  knowledge  of  the  floorplan  of  the  operating 
environment,  and  in  all  cases  the  robot  must  maintain  an  accurate  awareness  of  its  position  and 
orientation.  Differential  GPS  has  come  a  long  way  recently  in  satisfying  this  referencing  criteria  for 
outdoor  applications,  but  is  of  no  help  indoors  due  to  signal  blockage  by  the  building  structure  (Everett, 
1995).  Accordingly,  traditional  autonomous  navigation  techniques  are  of  limited  utility  for  scenarios 
where  the  requirement  exists  to  enter  previously  unexplored  structures  of  opportunity  as  the  need  arises. 

Teleoperated  systems,  on  the  other  hand,  permit  remote  operation  in  such  unknown  environments,  but 
conventionally  place  unacceptable  demands  on  the  operator.  For  example,  simply  driving  a  teleoperated 
platform  using  vehicle-based  video  feedback  is  no  trivial  matter,  and  can  be  stressful  and  fatiguing  even 
under  very  favorable  conditions.  Experience  gained  through  actual  use  (by  law  enforcement  and  military 
personnel)  of  conventional  teleoperated  devices  with  minimal  onboard  intelligence  has  revealed  other 
shortcomings  from  a  man/machine  interface  point  of  view.  Simply  put,  if  a  remote  operator  has  to 
master  simultaneous  manipulation  of  three  different  joysticks  (i.e.,  one  for  drive  and  steering,  another  for 
camera  pan  and  tilt,  and  possibly  yet  a  third  for  weapons  control),  the  chances  of  successfully  performing 
coordinated  actions  in  a  timely  fashion  are  minimal. 

Adding  to  the  difficulties  of  indoor  navigation  is  the  problem  of  receiving  reliable  data,  audio,  and  video 
communications  as  the  robot  explores  deeper  into  an  unknown  structure.  In  cluttered  environments, 
signal  blockage  and  multipath  reflections  can  quickly  degrade  performance  to  the  point  where 
communications  can  be  lost  altogether.  As  a  consequence,  it  is  quite  easy  for  a  teleoperated  robot  to  get 
disoriented  somewhere  inside  an  unfamiliar  building  and  be  unable  to  move  about  in  a  meaningful 
fashion,  or  perhaps  even  exit  back  to  the  street.  A  seamless  network  of  RE  repeaters  will  be  required  in 
such  cases  to  assure  clear  data  and  video  connections,  but  these  repeaters  need  to  be  strategically  placed 
along  the  path  of  transit  without  increasing  the  burden  on  the  operator  controlling  the  robot. 

For  these  reasons,  many  envisioned  potential  applications  of  robotic  response  vehicles  in  military 
scenarios  remain  impractical.  Accordingly,  this  development  effort  specifically  targets  three  critical 
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technological  needs:  1)  a  simplistic  graphical  user  interface  for  reflexive  teleoperated  control;  2)  the 
ability  to  build  an  accurate  world  model  on  the  fly  to  support  autonomous  operation;  and,  3)  robust  data 
and  video  communications  throughout  the  structure. 


2.  HUMAN-CENTERED  MAPPING 

Existing  autonomous  mobile  robots  typically  require  a  preconceived  and  very  detailed  map  (world 
model)  of  their  intended  operating  environment  for  path  planning  and  collision  avoidance  algorithms  in 
support  of  their  autonomous  navigation  needs,  but  most  law  enforcement  and  urban  warfare  scenarios 
preclude  the  availability  of  such  a  priori  information.  While  reflexive  teleoperated  control  concepts 
(Laird,  et  ah,  1990)  support  limited  remote  operation  of  tactical  mobile  robots  in  unexplored  urban 
environments,  there  is  the  additional  burden  of  keeping  track  of  the  robot’s  position  and  orientation. 
This  seemingly  trivial  task  can  quickly  become  very  tedious  (if  not  impossible)  due  to  the  limited 
information  readily  gleaned  from  an  onboard  video  camera  by  even  a  highly  skilled  operator  in  familiar 
settings.  The  situation  is  further  complicated  by  potential  video  signal  degradation,  poor  lighting,  little 
or  no  scene  contrast,  and  the  fact  that  the  user  probably  has  no  previous  experience  in  recognizing 
landmark  features  within  the  field-of-view. 


Figure  1.  Front  and  rear  views  of  ROBART  III,  an  intelligent  response  robot  capable  of  mapping  unexplored  structures. 

ROB  ART  in  (Figure  1)  specifically  addresses  this  issue  by  integrating  the  applicable  features  of 
reflexive  teleoperated  control  and  autonomous  control  to  produce  a  supervised  autonomous  system  that 
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can  quickly  explore  an  unknown  environment  with  minimal  human  oversight,  generating  a  world  model 
representation  that  supports  increasing  autonomy  of  operation. 

2. 1  Reflexive  Teleoperated  Control 

Easing  the  driving  burden  on  the  operator  was  a  major  force  behind  the  reflexive-teleoperated  control 
scheme  employed  on  a  previous  prototype,  ROBART  n  (Laird,  et  ah,  1990).  The  robot’s  numerous 
collision- avoidance  sensors  are  called  into  play  during  manual  operation  to  greatly  minimize  the 
possibility  of  operator  error.  The  commanded  speed  and  direction  of  the  platform  are  suitably  altered  as 
needed  by  the  onboard  processors  to  keep  the  robot  traveling  at  a  safe  speed  and  preclude  running  into 
obstructions.  Work  on  ROBART  ni  seeks  to  extend  this  concept  of  reflexive-teleoperation  into  the 
realm  of  sensor-assisted  camera  and  weapons  system  control.  A  user-friendly  graphical  interface  allows 
the  operator  to  issue  high-level  commands  to  the  robot,  with  low-level  control  details  addressed  in  real¬ 
time  by  onboard  intelligence  (Ciccimaro,  et  ah,  1998). 

2.1 .1  Reflexive  Mobility  Control 

The  operator  can  easily  control  platform  motion  by  clicking  on  special  behavioral  icons  depicted  on  the 
navigation  display  shown  in  Figure  2.  For  example,  selecting  a  wall-following  icon  to  either  side  of  the 
robot’s  own  icon  would  cause  the  platform  to  enter  wall-following  mode,  maintaining  its  current  lateral 
offset  from  the  indicated  wall  using  side-looking  sonar.  The  wall-following  icons  are  implemented 
under  Windows  95  as  long  vertical  command  buttons  situated  on  either  side  of  the  map  window  in  the 
lower  left  comer  of  the  screen.  The  platform’s  speed  of  advance  is  initially  dictated  by  onboard 
intelligence  as  a  function  of  perceived  congestion,  but  can  be  adjusted  to  a  certain  extent  by  the  operator 
using  the  arrow  icons  in  the  Mobility  Control  window  shown  in  the  lower  right  corner. 

Two  additional  wall  segment  icons  are  seen  above  the  map  in  the  form  of  short-length  horizontal 
command  buttons.  The  open  spaces  between  these  graphical  depictions  of  wall  structures  represent  three 
potential  doorways:  one  directly  ahead  of  the  robot  and  one  on  either  side.  By  clicking  in  one  of  these 
doorway  icons,  the  robot  is  instructed  to  seek  out  and  enter  the  next  encountered  location  of  that  type  of 
door  along  its  current  path.  The  head-mounted  surveillance  camera  automatically  pans  to  the  correct 
position  to  allow  the  remote  operator  to  monitor  the  doorway  detection  and  penetration  functions  as  they 
occur.  For  the  example  illustrated  in  Figure  2,  the  platform  is  following  the  left  wall  while  looking  for  a 
door  off  to  the  left  (as  indicated  by  the  highlight  box  shown  in  the  selected  doorway  icon,  and  the 
associated  text  displayed  in  the  System  Status  window  above  the  map). 
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Figure  2.  Highlighted  icons  to  the  left  of  the  Map  Display  Window  (lower  left  corner)  indicate  ROBART  III  is  following  the 
wall  on  its  left,  looking  for  an  open  doorway  on  the  left. 


2.1 .2  Reflexive  Non-Lethal  Weapon  Control 

A  key  objective  of  this  development  effort  is  to  extend  the  concept  of  reflexive  teleoperation  to  include 
automated  non-lethal  weapon  control  in  response  to  a  perceived  threat.  Extremely  robust  intruder 
detection  and  assessment  capabilities  with  minimal  nuisance  alarms  are  achieved  through  intelligent 
fusion  of  a  multitude  of  inputs  from  various  onboard  motion  sensors.  The  intruder  detection  algorithm 
operates  upon  the  output  of  a  line-oriented  Video  Motion  Detector  (VMD)  and  a  360-degree  array  of 
passive-infrared  sensors  configured  as  a  collar  just  below  the  head  as  shown  in  Figure  1.  The  intruder 
bearing  in  robot  coordinates  can  be  determined  from  the  identity  of  the  active  passive-infrared  array 
element,  and  used  to  pan  the  head-mounted  surveillance  camera  to  the  center  of  any  zone  with  suspected 
intruder  activity.  The  VMD  track  output  is  then  used  to  keep  the  intruder  in  the  center  of  the  visual 
field. 

A  combination  of  robot  head  and  body  movement  are  employed  to  keep  the  target  centered  if  the  head  is 
about  to  reach  its  maximum  pan  limit  (±100  degrees)  relative  to  the  robot.  When  a  head  pan  angle  of 
±90  degrees  is  exceeded,  the  robot’s  body  will  automatically  pivot  in  place  towards  the  target,  while  the 
head  smoothly  moves  at  the  same  rotational  speed  in  the  opposite  direction  to  keep  the  target  in  the 
center  of  the  visual  field.  This  coordinated  action  provides  unlimited  (i.e.,  >  360  degrees)  pan  coverage, 
and  frees  the  operator  from  having  to  manipulate  separate  joysticks  to  manually  achieve  the  same  effect. 
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An  automatic  Gun  Track  Mode  can  be  activated  by  the  host  while  ROB  ART  HI  is  in  Motion  Tracking 
Mode,  eausing  the  robot  to  re-eenter  its  head  and  turn  to  direetly  faee  the  deteeted  threat.  The  robot  then 
beeomes  stationary  while  the  gun  begins  to  traek  the  target  using  relative  bearing  information  from  the 
VMD  and  range  information  from  the  head-mounted  ultrasonie  sonar  array.  At  this  point,  a  non-lethal 
weapon  response  ean  be  invoked  if  warranted.  While  ROBART  III  is  responsible  for  automatie  target 
traeking,  the  aetual  firing  deeision  is  made  by  the  human  operator  upon  seeing  the  weapon’s  laser  spot 
look  on  a  validated  target  in  the  surveillanoe  oamera  video. 

The  prinoiple  non-lethal  response  system  ourrently  inoorporated  on  ROBART  HI  is  a  six-barreled 
pneumatioally-powered  Gatling  gun  (see  Figure  3)  oapable  of  firing  a  variety  of  3/16-inoh  diameter 
projeotiles  (i.e.,  simulated  tranquilizer  darts  or  plastio  bullets  manufaotured  from  Teflon  or  Delrin). 
Projeotiles  are  expelled  at  a  high  velooity  from  12-inoh  barrels  by  a  release  of  oompressed  air  from  a 
pressurized  aooumulator  at  the  rear  of  the  gun  assembly.  In  addition  to  single-shot  mode,  all  six  barrels 
ean  be  fired  in  rapid  suooession  (i.e.,  approximately  1.5  seoonds).  The  main  air  bottle  is  automatioally 
reoharged  by  a  small  12-volt  reoiprooating  oompressor  mounted  in  the  robot’s  base  (Cieeimaro,  1998). 


Figure  3.  A  six-barrel  Gatling-style  tranquilizer  dart  gun  is  employed  on  ROBART  111  to  illustrate  one  of  a  multitude  of 
automated  non-lethal  response  options. 


2.2  Autonomous  Control 

As  previously  stated,  some  type  of  world  modeling  seheme  is  required  to  evolve  from  reflexive 
teleoperated  control  to  supervised  autonomous  control.  The  exploratory  mapping  of  unknown 
struetures  into  sueh  a  model  benefits  signifieantly  when  the  interpretation  of  raw  sensor  data  is 
augmented  by  simultaneous  supervisory  input  from  the  human  operator.  The  end  result  of  sueh  a 
"human-eentered  mapping"  strategy  is  a  mueh  faster  and  more  aeeurate  generation  of  objeet 
representations  (relative  to  eonventional  sensor-only  eonfigurations),  partieularly  valuable  when  there  is 
no  a  priori  information  available  to  the  system.  In  a  nutshell,  the  robot  ean  enter  and  explore  an 
unknown  spaee,  building  a  valid  model  representation  on  the  fly,  while  repeatedly  refereneing  itself  in 
the  proeess  to  null  out  aeeumulated  dead-reekoning  errors. 
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For  example,  a  mathematical  line-fit  analysis  is  typically  used  to  detect  the  presence  of  a  suitable  wall¬ 
like  structure  that  can  be  used  as  a  navigational  reference,  as  will  be  discussed  in  the  next  section.  With 
just  minimal  human  input,  the  robot  doesn’t  just  think  it  sees  a  wall,  it  knows  it  sees  a  wall.  The 
operator,  for  example,  upon  first  entering  a  building  could  instruct  the  robot  using  simple  commands 
like:  1)  “follow  the  wall  on  your  left;”  and,  2)  “enter  the  next  doorway  on  the  left.”  Such  high-level 
direction  is  provided  by  clicking  on  screen  icons  as  depicted  earlier  in  Figure  2.  In  addition  to  directing 
the  robot’s  immediate  behavior,  however,  these  same  commands  can  also  provide  valuable  information 
to  the  world  modeling  algorithm,  as  explained  below. 

Prior  to  the  start  of  building  exploration,  the  world  model  is  initialized  as  a  two-dimensional  dynamic 
array  with  all  cells  initially  marked  as  unknown.  (An  unknown  cell  is  treated  as  potentially  traversable, 
but  more  likely  to  be  occupied  than  confirmed  free  space.)  If  some  specific  subset  of  the  side-looking 
sonar  data  can  be  positively  identified  from  the  outset  as  a  wall-like  structure,  it  can  be  unambiguously 
modeled  right  away  as  a  confirmed  wall  without  the  need  for  statistical  representation.  This  makes  the 
resulting  world  model  much  less  ambiguous  in  that  key  navigational  features  are  expressly  flagged  as 
such,  and  the  find-path  search  of  the  path  planner  can  be  optimized  accordingly. 

By  way  of  further  illustration,  it’s  also  quite  probable  that  an  open  doorway  will  be  detected  during 
transit  associated  with  the  aforementioned  example:  (i.e.,  “enter  the  next  doorway  on  your  left”).  A 
doorway  represents  another  distinctive  feature  of  the  real  world  that  can  be  exploited  in  the  generation  of 
the  model,  provided  there  is  some  suitable  means  of  positively  identifying  such  (i.e.,  by  robot  sensors, 
human  observance,  or  in  this  case,  both).  Here  the  operator’s  “heads  up”  for  the  robot  to  expect  a 
doorway,  based  on  his  or  her  assessment  of  the  video  telemetry,  facilitates  a  high-confidence 
classification  of  the  next  perceived  wall  opening  on  the  indicated  side  as  a  doorway  feature.  The 
resulting  model  representation  of  a  doorway  represents  a  portal  of  guaranteed  passage,  whereas  a 
confirmed  wall  is  always  interpreted  as  a  non-traversable  boundary.  Saving  information  describing  the 
location  and  orientation  of  detected  doorways  is  readily  accomplished  by  assigning  each  of  the  two 
certainty-grid  cells  associated  with  the  perceived  locations  of  the  door  edges  a  unique  “special-case” 
value  that  flags  the  feature  later  for  the  path  planner. 

Since  doorways  are  openings  in  walls,  additional  valuable  information  can  be  inferred  by  assuming  there 
must  be  an  associated  wall  segment  running  along  an  imaginary  line  defined  by  the  left  and  right  sides  of 
a  door.  Accordingly,  the  entire  line  of  certainty-grid  cells  defined  by  the  two  specially  marked  door-edge 
cells  in  the  model  are  encoded  to  represent  the  location  of  a  potential  wall.  This  particular  inference 
(i.e.,  where  there’s  a  door,  there  must  be  a  wall)  is  not  all  that  significant  in  the  foregoing  example,  since 
the  door  was  discovered  by  following  a  known  wall  in  the  first  place.  But  in  the  case  where  the  operator 
invokes  a  door-ahead  search  (i.e.,  seek  out  a  door  opening  directly  ahead),  the  situation  is  essentially 
reversed,  in  that  the  exploring  robot  initially  has  no  knowledge  of  the  associated  wall. 

Such  potential  wall  representations  extend  to  the  map  boundaries  in  the  direction  of  unexplored 
{unknown)  territory,  but  would  terminate  upon  intersection  with  any  previously  identified  features  such 
as  trails,  confirmed  walls,  and  doorways.  From  the  path-planner’s  perspective,  the  cost  of  crossing  this 
potential  wall  representation  is  higher  than  the  cost  of  traversing  unknown  floor  space,  but  less  than  the 
cost  of  traversing  a  confirmed  wall.  A  traversal  through  the  associated  doorway,  of  course,  has  zero 
cost. 
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2.3  Orthogonal  Navigation 


The  Achilles’  Heel  of  any  world  modeling  scheme,  however,  is  accurate  positional  referencing  in  real¬ 
time  by  the  moving  platform.  Since  all  sensor  data  is  taken  relative  to  the  robot’s  location  and 
orientation,  the  accuracy  (and  usefulness)  of  the  model  quickly  degrades  as  the  robot  becomes 
disoriented.  While  wall  following  is  a  very  powerful  tool  in  and  of  itself  for  determining  the  relative 
offset  and  heading  of  the  robot,  conventional  schemes  normally  assume  some  a  priori  information  about 
the  wall  in  the  first  place  to  facilitate  its  utility  as  a  navigational  reference.  In  short,  a  relative  fix  with 
respect  to  an  unknown  entity  does  not  yield  an  unambiguous  absolute  solution,  for  obvious  reasons.  The 
problem,  of  course,  is  that  no  such  advanced  information  on  the  absolute  position  and  orientation  of 
walls  is  available  for  previously  unexplored  environments. 

The  Ortho-Mode  navigation  scheme  employed  on  ROBART  HI  gets  around  this  problem  by  exploiting 
the  orthogonal  nature  of  most  building  structures,  where  walls  for  the  most  part  are  either  parallel  or 
orthogonal.  Ortho-mode  uses  the  input  from  an  inexpensive  ($50)  Vector-2X  magnetoinductive  compass 
(manufactured  by  Precision  Navigation  of  Mountain  View,  CA)  to  address  the  issue  of  absolute  wall 
orientation.  The  accuracy  of  the  compass  (optimistically  advertised  as  ±2  degrees)  need  only  be  good 
enough  to  resolve  the  ambiguity  of  which  of  four  possible  wall  orientations  the  robot  has  encountered. 
This  information  is  stored  in  the  model  in  conjunction  with  the  wall  representation  (i.e.,  wall  segment 
running  north-south,  or  wall  segment  running  east-west),  in  arbitrary  building  coordinates.  The  precise 
heading  of  the  vehicle  (in  building  coordinates)  is  then  mathematically  derived  using  sonar  data  taken 
from  the  wall  surface  as  the  robot  moves. 

In  the  simplest  type  of  wall  following,  the  robot  measures  the  range  to  the  indicated  side  and  reflexively 
drifts  closer  or  further  from  the  wall  in  order  to  maintain  a  pre-specified  offset,  reacting  to  each  range 
value  as  soon  as  it  arrives.  A  more  elegant  variation  of  this  approach  throws  away  any  “noisy”  range 
data  that  does  not  fall  within  an  expected  window  of  acceptance,  smoothing  the  robot’s  trajectory 
accordingly.  In  an  even  better  instantiation,  the  robot  is  told  its  starting  location,  lateral  offset,  and 
desired  end  point,  but  uses  a  number  of  sonar  readings  to  obtain  a  line  fit  that  approximates  the  wall 
orientation  and  displacement  before  reacting  (Holland,  et  ah,  1995).  The  robot  then  periodically  corrects 
its  heading  and  lateral  position  using  the  filtered  data  solution,  with  much  better  results. 

This  latter  approach  is  employed  in  the  Ortho-Mode  scheme,  only  ROBART  HI  is  simply  told  to  follow 
the  wall  to  its  left  or  right  without  any  amplifying  parameters  (i.e.,  wall  orientation  and  offset).  The 
robot  collects  a  number  of  side-sonar  readings  to  obtain  a  least-squares  line  fit  for  the  wall  (Figure  4), 
which  is  used  to  maintain  whatever  stand-off  distance  was  in  effect  at  time  of  invocation.  (Alternatively, 
the  desired  lateral  offset  can  be  predefined.) 

The  as  yet  unknown  orientation  of  the  wall  is  then  derived  from  the  magnetic  compass  heading  once 
wall-following  has  stabilized,  being  automatically  set  to  either  0,  90,  180  or  270  degrees  (with  respect  to 
an  arbitrary  building  North  of  0  degrees).  The  choice  is  simply  based  on  which  of  these  four  possibilities 
best  matches  the  measured  compass  reading  within  a  pre-specified  window  of  acceptance.  This 
reference  value  is  stored  in  the  model,  and  also  used  with  subsequent  line-fit  solutions  to  precisely 
correct  the  robot's  absolute  heading  register  in  the  reference  frame  of  the  building. 
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Figure  4.  Plot  of  measured  ultrasonic  range  data  during  mapping  of  a  hallway  and  side  passageway.  The  dotted  lines  show 
where  the  orientation  of  Perceived  Walls  have  been  “snapped”  into  precise  alignment  by  the  Ortho-Mode  algorithm. 


3.  DISTRIBUTED  MASTER  /  SLAVE  NETWORK 

To  address  the  communication  degradation  problem,  a  master/slave  network  can  be  instituted  in 
conjunction  with  building  exploration  by  automatically  deploying  a  number  of  slave  robots  to  act  as  a 
distributed  array  of  mobile  repeaters.  These  slaves  blindly  follow  ROB  ART  in  in  trailing  convoy 
fashion,  being  deployed  and  recovered  automatically  so  the  remote  operator  need  only  provide 
supervisory  control  of  the  master.  The  slave  robots  can  perform  a  multitude  of  assigned  functions,  to 
include  acting  as  a  distributed  video  and  motion  sensor  network  and/or  RF  repeaters. 

A  fleet  of  ten  Lynxmotion  Hexapod  II  walking  robots  (six-legged,  twelve-servo  hexapods  featuring  two 
degrees-of-freedom  per  leg)  are  currently  used  to  illustrate  the  feasibility  of  the  master/slave  network 
(Figure  5).  This  temporary  demonstration  platform  was  selected  not  on  the  basis  of  its  future  operational 
suitability,  but  because  of  its  low-cost,  ready  availability,  and  limited  payload  potential,  which  forced 
designers  to  seek  out  new  solutions  in  the  form  of  small  and  lightweight  componentry.  For  example,  a 
combination  of  miniature  ultrasonic  and  near-infrared  ranging  sensors  are  employed  for  collision 
avoidance.  Short-range  detection  (4  to  31  inches)  is  handled  by  five  Sharp  DP2D12  near-infrared 
triangulation  ranging  sensors:  two  side-looking  units  and  a  fan-shaped  array  of  three  covering  the 
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forward  sector.  Long-range  obstacle  detection  (1  to  10  feet)  is  accomplished  with  a  forward-looking 
EDP  Mini-A  Ultrasonic  SonaSwitch,  a  self-contained  Polaroid-based  sonar  module  with  a  0-  to  5-volt 
analog  output. 


Figure  5.  Two  of  eight  slave  robots  in  the  early  stages  of  development.  The  slave  on  the  left  is  equipped  with  a  near-infrared 
beacon-tracking  sensor  while  the  slave  on  the  right  is  capable  of  intruder  detection  and  response  with  a  triple-barrel  CO2  BB 
cannon. 

The  small  slave  robots  perform  collision-avoidance,  wall-following,  and  doorway-detection  routines 
using  algorithms  similar  to  those  running  on  ROB  ART  III.  For  ah  other  tasks,  the  slaves  react  in 
simplistic  fashion  to  information  that  has  been  gathered  and  preprocessed  by  the  master  robot  to 
minimize  required  complexity.  For  example,  let’s  assume  the  master  has  just  located  an  open 
passageway  leading  to  an  adjoining  room  while  executing  a  wall-following  command.  A  trailing  slave, 
commanded  to  enter  this  doorway  and  take  up  station,  breaks  from  the  convoy  and  simply  follows  the 
known  clear  path  along  the  indicated  wall  until  the  next  opening  is  detected.  The  individual  slaves  can 
thus  autonomously  execute  a  small  library  of  behavior  primitives,  but  lacking  a  world  model,  have  no 
absolute  awareness  of  their  own  locations. 

3.1  Slave  Functions 

ROBART  in  controls  ah  management  of  the  slaves  and  their  various  functions  in  transparent  fashion, 
leaving  the  remote  operator  to  concentrate  on  supervising  just  the  master  and  not  a  collection  of 
interacting  robots.  The  slaves  can  perform  a  multitude  of  assigned  functions,  to  include:  1)  following  in 
trail  (convoying);  2)  acting  as  distributed  video  and  motion  sensor  network;  and,  3)  providing  a 
seamless  RF-relay  capability  (data,  audio,  and  video)  throughout  the  structure. 
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3.1.1  Convoying 


An  optical  beacon-following  scheme  has  been  implemented  as  a  simple  way  for  multiple  small  robots  to 
see  and  identify  each  other  when  convoying  in  single  file.  The  master  and  slave  robots  are  each  outfitted 
with  a  rear-facing  180-degree  array  of  940-nanometer  near- infrared  emitters  to  serve  as  a  homing  beacon 
that  can  be  detected  out  to  a  maximum  range  of  about  ten  feet.  Each  slave  is  further  equipped  with  a 
pair  of  Sharp  GPl  U58X  near- infrared  detector  modules  mounted  side-by-side  on  top  of  a  small  pan  unit 
at  the  front  of  the  robot.  The  sensor  field-of-view  has  been  optically  limited  to  about  a  30-degree  cone  to 
control  the  overlap  in  coverage  so  the  beacon  must  be  directly  centered  before  it  becomes  detectable  by 
both  channels. 

The  Sharp  sensors  have  a  built-in  bandpass  filter  centered  at  40  kiloHertz  to  minimize  ambient 
interference  from  other  near-infrared  sources.  Since  the  digital  output  stage  of  the  module  is  AC 
coupled,  the  output  pin  goes  low  when  a  40-kHz  signal  is  detected,  but  returns  to  its  normally  high  state 
in  about  200  milliseconds.  To  work  around  this  problem,  the  40-kHz  carrier  frequency  of  the  near- 
infrared  beacon  is  modulated  at  1  kHz.  The  “beacon- acquired”  detector  output  subsequently  becomes  a 
steady  1-kHz  square  wave,  which  is  converted  to  a  binary  status  bit  using  a  1-kHz  tone  decoder. 

A  slave  robot  placed  by  the  master  in  Follow  Mode  will  scan  back  and  forth  in  azimuth  until  one  of  the 
Sharp  modules  sees  a  valid  beacon  signal.  The  sensor  pair  is  panned  in  the  appropriate  direction  until 
both  sensors  can  see  the  emitter.  With  the  pan  unit  thus  locked  onto  the  master  robot,  the  slave  begins 
walking  forward,  turning  in  the  process  to  align  its  body  with  the  direction  indicated  by  the  beacon 
sensor.  A  SonaSwitch  ranging  module  mounted  on  the  pan  unit  directly  above  the  Sharp  sensors 
measures  the  longitudinal  separation  between  the  front  of  the  slave  and  a  cooperative  acoustical  reflector 
incorporated  into  the  near-infrared  beacon  assembly.  As  the  master  moves  forward,  the  slave  will  then 
tag  along  behind  at  a  prespecified  offset  interval. 

To  retrieve  the  network  of  deployed  slaves,  the  master  retraces  its  path  and  individually  directs  them  at 
the  proper  time  to  look  for  and  follow  the  optical  beacon.  To  preclude  collisions  as  a  recalled  slave 
attempts  to  get  back  into  a  trailing  position,  the  last  robot  (including  the  master)  in  the  convoy  must  be 
uniquely  identifiable.  This  is  accomplished  by  changing  the  modulation  frequency  of  the  beacon 
emissions  from  the  trailing  robot  to  2  kiloHertz  using  a  digital  potentiometer.  The  slave  robots 
discriminate  between  the  two  possible  modes  by  dynamically  reconfiguring  their  tone  detectors  to 
identify  which  signal  is  being  received. 

3.1 .2  Remote  Intruder  Detection  and  Response 

In  order  to  deter  an  intruder  from  playing  “hide-and-seek”  with  the  master,  slave  robots  can  be  left 
behind  at  strategic  vantage  points  (i.e.,  open  doorways,  hallway  intersections)  to  watch  its  back,  alerting 
the  master  (via  the  interconnecting  RF  link)  to  any  perceived  motion  detected  by  their  sensors.  This 
remote  intruder  detection  is  accomplished  primarily  with  passive-infrared  motion  sensors,  augmented  to 
some  extent  by  the  ultrasonic  and  near-infrared  collision  avoidance  sensors.  A  miniature  video  camera 
mounted  on  the  beacon-tracking  pan  unit  supports  remote  video  surveillance  if  desired  by  the  operator. 
The  master  robot  saves  the  deployed  location  and  orientation  of  all  slaves  in  the  world  model,  and  given 
the  relative  bearing  (in  slave  coordinates)  of  the  perceived  disturbance,  can  easily  plan  a  path  back  to  the 
general  location  of  the  suspected  intruder  for  further  investigation. 


Ciccimaro-1 


11 


The  feasibility  of  a  supervised  (via  remote  video)  non-lethal  response  from  a  slave  robot  has  been 
illustrated  using  a  triple-barrel  BB  cannon  which  can  be  fired  on  command  by  the  operator  (Figure  5).  A 
12-gram  CO2  cylinder  is  carried  onboard  as  a  lightweight  compressed  gas  source.  All  three  barrels  fire 
at  once,  gravity  fed  from  an  integral  75-round  magazine,  and  gun  rotation  is  achieved  using  a  pulse- 
proportional  servo  designed  for  radio-controlled  toys. 

3.1.3  Distributed  Video 

Availability  of  distributed  video  sensors  throughout  the  structure  can  significantly  contribute  to  the 
remote  intruder  detection  and  response  function  as  discussed  above.  Another  advantage  is  realized  with 
respect  to  reflexive  teleoperation  of  the  master,  as  video  relayed  back  to  the  remote  operator  from 
ROB  ART  in’s  onboard  surveillance  camera  can  be  somewhat  limited  in  coverage.  Even  with  the 
relatively  wide  90-degree  camera  field-of-view,  the  lack  of  a  tilt  axis  essentially  precludes  seeing  the 
floor  anywhere  within  five  feet  of  the  robot.  This  limitation  can  seriously  impair  the  remote  operator’s 
situational  awareness  when  maneuvering  in  tight  quarters.  An  over-the-shoulder  view  from  a  trailing 
slave  robot  can  provide  a  valuable  perspective  in  such  situations.  Accordingly,  the  operator  can  switch 
views  from  the  master’s  onboard  camera  to  that  of  any  individual  slave,  even  while  in  motion,  since  the 
automatic  collision  avoidance  afforded  by  reflexive  teleoperation  precludes  the  need  to  closely  monitor 
video  in  the  first  place. 

The  slave  robots  on  command  can  transmit  video  (one  at  a  time)  from  their  own  onboard  cameras  over  a 
separate  low-power  analog  channel  (Figure  6)  to  the  master.  The  master  specifies  which  (if  any)  slave 
has  access  to  this  channel,  and  that  slave  responds  by  turning  on  its  onboard  transmitter,  a  Micro  Video 
MV915VTx,  one  of  the  smallest  wireless  video  transmitters  for  the  900-MHz  Industrial,  Scientific  and 
Medical  (ISM)  band.  All  other  slaves  respond  by  ensuring  their  transmitters  are  powered  down.  A  two- 
channel  video  multiplexor  onboard  ROB  ART  HI  is  used  to  select  which  imagery  is  relayed  to  the  remote 
operator  over  the  high-power  link  (i.e.,  from  the  master’s  own  head-mounted  surveillance  camera,  or 
relayed  video  from  the  designated  slave  robot.) 

A  Trango  Eagle  1000  wireless  video/audio  system  has  recently  been  installed  on  the  master  for  much 
improved  signal  quality  and  effective  range  back  to  the  remote  operator.  (The  previous  ProVideo  ITCS- 
50T  transmitter  had  a  maximum  indoor  range  of  150  feet,  but  with  clear  transmissions  typically  being 
received  only  as  far  as  50  feet.)  The  Eagle  1000  provides  exceptional  line-of-sight  transmission  ranges 
of  3-4  miles  outdoors,  with  indoor  ranges  averaging  300-1000  feet,  and  can  also  pass  two  binary  alarm 
signals  to  its  dedicated  receiver  on  any  of  four  user- selectable  video  channels. 
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Figure  6.  Micro  Video  MV915VTx  wireless  video  transmitter  (left)  and  BAV  MVC  058  pinhole  camera  (center)  form  the 
basis  of  the  slave-to-robot  analog  video  link. 


3.1.4  Communications  Relay 

Non-line-of-sight  communication  between  the  master  and  slaves  is  currently  implemented  using  a  set  of 
STI  spread- spectrum  RF  modems,  which  can  be  configured  for  various  forms  of  wireless  network 
topologies  such  as  point-to-point  and  broadcast  mode.  The  master/slave  data  link  is  a  hybrid  of  both, 
allowing  the  master  to  communicate  with  any  of  the  slave  robots,  while  the  slaves  can  only  communicate 
with  the  master  and  not  with  each  other.  As  the  exploring  robot  probes  deeper  into  an  unexplored 
building,  video  and  data  communication  can  become  seriously  degraded  or  lost  altogether. 

When  this  occurs,  the  master  robot  can  dispatch  an  RF  slave  relay  to  restore  a  clear  communications  link 
between  the  operator  and  master.  After  a  predetermined  time  with  no  incoming  operator  commands,  the 
master  will  generate  a  communication  timeout  and  automatically  deploy  a  dedicated  slave  relay  in  an 
attempt  to  regain  communications  with  the  remote  operator.  A  slave  deployed  as  an  RF  relay  is  capable 
of  communicating  with  both  the  master  and  remote  operator,  as  is  seen  in  the  communication  topology 
of  Figure  7.  The  principle  limitation  of  this  current  RF  relay  scheme,  aside  from  the  fact  that  much  more 
elegant  off-the-shelf  modem  technology  has  been  recently  introduced,  is  that  it  applies  only  to  the  data 
channel  and  does  not  address  video  or  audio. 

3.1 .5  Expendable  Point  Man 

Since  the  master  response  robot  represents  the  one  piece  of  equipment  an  assault  team  depends  on  most 
for  remotely  sweeping  an  unexplored  structure  for  potential  threats,  a  slave  robot  can  be  ordered  to  “take 
point”  if  hostile  action  is  expected.  This  strategy  would  allow  the  inexpensive  (i.e.  expendable)  slave 
robot  to  discover  or  wander  into  any  danger  and  expose  the  threat,  giving  the  master  extra  time  to  react 
and/or  alert  friendly  forces.  This  function  has  not  yet  been  implemented  in  the  present  system,  as  it 
requires  some  additional  navigational  capability  (i.e.,  beyond  beacon  following)  on  the  part  of  the  slave. 
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Figure  7.  Master  /  Slave  data  communication  topology  when  direct  communication  between  the  master  and  remote  operator 

is  no  longer  possible 


3.2  Digital  Video  Upgrade 

Audio  and  composite  video  signals  are  eurrently  relayed  to  the  operator  via  a  high-power  analog  video 
transmitter,  but  an  experimental  brassboard  digital  link  has  been  developed  to  allow  for  eventual 
integration  of  RS-232  data,  audio,  and  eompressed  video  into  a  single  format  to  reduee  eomplexity,  size, 
weight,  and  power  consumption.  Several  issues  were  eonsidered  toward  realization  of  an  effeetive  video 
stream  ehannel: 

•  Loss  of  information  eontent; 

•  Coding  and  deeoding  (CODEC)  effieieney  for  eompression  of  2D  image  stream  data; 

•  Minimization  of  radiation  emissions  for  radio  frequeney  links; 

•  Power  eonsumption; 

•  Reliable  eommunieation  strategy  for  a  eonstellation  of  mobile  robots. 

The  ehosen  solution  as  implemented  (Figure  8)  provides  eompression  of  the  video  stream,  transmission 
of  the  eompressed  data  over  a  band-limited  serial  RF  ehannel,  and  deeompression  at  the  reeeiver. 

Three  image  eompression  algorithms  were  eonsidered:  1)  diserete  eosine  transformation  (DCT); 
2)  Moving  Pietures  Experts  Group  (MPEG);  and,  3)  a  wavelet-based  eompression  seheme  (Grossman  & 
Morlett,  1984).  A  survey  of  digital  television  industry  standard  praetiees  and  the  status  of  the  evolving 
IEEE-1394-oompliant  produets  was  also  eondueted  to  ensure  appropriate  exploitation  of  eommereial  off- 
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the-shelf  options.  A  conscious  effort  was  made  to  implement  successful  algorithms  in  hardware  where 
possible.  To  this  end  we  selected  the  bi-orthogonal  wavelet  transform  filter  based  on  work  by  Grossman 
and  Morlet  (1984)  and  sub-band  CODEC  as  implemented  in  the  Analog  Devices  ADV611  chip.  Details 
about  the  hardware  implementation  of  the  CODEC  can  be  found  in  Analog  Devices  literature  (AD, 
1998). 


Figure  8.  Experimental  brassboard  digital  link  which  will  allow  for  eventually  integration  of  RS-232  data,  audio,  and 
compressed  video  into  a  single  format. 

Referring  now  to  Eigure  9,  each  video  frame  is  digitized  by  a  Philips  SAA7111A  chip,  whereupon  the 
digital  image  is  transformed  from  video  spatial  data  to  spatial-frequency-filtered  sub-bands.  The  result 
is  then  fed  into  a  run-length  encoder  and  Huffman  encoder  module.  An  Analog  Devices  ADSP2185 
digital  signal  processing  (DSP)  chip  is  utilized  to  output  a  synchronous  bit  stream  to  a  communication 
device.  The  DSP  chip  also  provides  bin  counting  feedback  to  the  CODEC  module,  so  the  degree  of 
compression  accommodates  information  quality  at  the  synchronous  data-rate  bit  clock.  The  system  can 
process  30  frames  per  second,  but  is  intentionally  limited  in  this  implementation  to  two  frames  per 
second  in  order  to  preserve  information  content  in  the  image  stream  from  general  mission  scenarios. 
The  image  stream  as  viewed  at  the  receiver  is  relatively  free  of  “blocky  or  blurred”  artifacts  at  two 
frames  per  second,  and  is  comfortably  formatted  to  accommodate  transfer  over  115-kilobaud  RE 
modems. 


Figure  9.  The  video  link  is  designed  to  provide  a  balance  between  keeping  the  operator  informed  and  meeting  physical 
bandwidth  limitations. 

In  recognition  of  requirements  for  minimized  RE  emissions  during  some  types  of  military  surveillance 
missions,  and  complexities  associated  with  individually  relaying  multiple  signals  (i.e.,  audio,  video. 
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data),  several  data  streams  now  share  a  single  bandwidth-limited  communications  channel.  A  universal 
asynchronous  receiver/transmitter  (UART)  was  implemented  in  a  pair  of  field-programmable-gate-arrays 
(FPGAs)  to  preserve  the  flexibility  to  later  integrate  other  types  of  information  content  into  the  serial  bit 
stream.  Future  instantiations  of  this  video  pipe  design  will  include  realization  of  a  formatter  placed 
within  the  FPGA.  The  formatter  will  provide  capability  to  integrate  other  useful  measurements  such  as 
temperature,  pressure,  global  positioning  system  data,  spectrometer  data,  and  other  physical  data  from 
sensors  into  the  data  stream. 

4.  CONCLUSION 

This  paper  provides  a  status  update  on  the  implementation  of  a  prototype  tactical/security  response  robot 
capable  of  exploration  in  unknown  structures.  A  human-centered  mapping  strategy  ensures  accurate 
first-time  interpretation  of  navigational  landmarks  as  the  robot  builds  its  world  model,  while  orthogonal 
navigation  exploits  the  fact  that  the  majority  of  man-made  structures  are  characterized  by  parallel  and 
orthogonal  walls.  A  distributed  surveillance  and  communication  network  can  be  transparently  deployed 
in  the  form  of  multiple  autonomous  slave  robots  for  increased  effectiveness  while  exploring  deep  into 
unknown  structures. 

Future  efforts  will  focus  on  a  number  of  potential  upgrades  to  the  current  “feasibility-demonstration” 
prototype: 

•  Implement  a  reflexive  teloperated  pursuit  mode  that  slaves  platform  steering  to  the  weapon 
pan  axis,  allowing  the  robot  to  automatically  “move  to  where  the  gun  is  pointing;” 

•  Implement  a  small  inexpensive  laser  rangefinder  in  place  of  the  outdated  Hamamatsu 
triangulation  ranging  sensor  for  detecting  open  doorways; 

•  Move  the  world  modeling  software  to  a  single-board  Pentium-based  processor  onboard  the 
master  robot; 

•  Substitute  a  digital  frame  grabber  for  the  line  grabber  currently  running  the  video  motion 
detection  algorithms; 

•  Develop  a  more  robust  (i.e.,  faster,  bigger  payload  capacity)  wheeled  version  of  the  slave 
robots  to  replace  the  mobility- limited  hexapods; 

•  Incorporate  an  optimized  integrated  data/video  link  to  reduce  complexity  and  facilitate  full 
relay  capabilities  for  data,  audio,  and  compressed  video. 
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